The present disclosure relates to stochastic gradient descent algorithms and, more specifically, to methods, systems and computer program products for performing an asynchronous stochastic gradient descent.
Stochastic gradient descent is a gradient descent optimization method for minimizing an objective function that may be non-convex. Stochastic gradient descent is a popular algorithm for training a wide range of models in machine learning, including support vector machines, logistic regression and graphical models. One popular stochastic gradient descent algorithm is the least mean squares (LMS) adaptive filter.
Asynchronous stochastic gradient descent is commonly used to train deep neural networks (DNNs), which are behind many breakthroughs in machine learning in a variety of areas.